The traditional sparse coding image classification algorithms extract single type features, ignore the spatial structure information of the images, and can not make full use of the feature topological structure information in feature coding. In order to solve the problems, a image classification algorithm based on multi-scale feature fusion and Hessian Sparse Coding (HSC) was proposed. Firstly, the image was divided into sub-regions with multi-scale spatial pyramid. Secondly, the Histogram of Oriented Gradient (HOG) and Scale-Invariant Feature Transform (SIFT) were effectively merged in each subspace layer. Then, in order to make full use of the feature topology information, the second order Hessian energy function was introduced to the traditional sparse coding target function as a regularization term. Finally, Support Vector Machine (SVM) was used to classify the images. The experimental results on dataset Scene15 show that, the accuracy of HSC is 3-5 percentage points higher than that of Locality-constrained Linear Coding (LLC), while it is 1-3 percentage points higher than that of Support Discrimination Dictionary Learning (SDDL) and other comparative methods. Time-consuming experimental results on dataset Caltech101 show that, the time-consuming of HSC is about 40% less than that of the Multiple Kernel Learning Sparse Coding (MKLSC). The proposed HSC can effectively improve the accuracy of image classification, and its efficiency is also better than the contrast algorithms.